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Program description 


Lessons in Character is designed to promote elementary and 
middle school students’ knowledge about core character 


stories, writing activities, and class projects. Teachers introduce 
the theme with a story that shows a value in action; students 



education values and, through that knowledge, shape children’s then engage that topic with a variety of activities. The program 

positive behaviors and support academic success. It consists also includes daily oral language development and weekly writ- 

of 24 lessons organized around weekly themes, taught through ing assignments, optional parts of the program’s implementation 



Research One study of Lessons in Character met the What Works Clearing- attending schools in three southern states. The studies exam- 
house (WWC) evidence standards, and a second study met ined results on students’ behavior; knowledge, attitudes, and 

WWC standards with reservations. The two studies included values; and academic achievement. 1 

a total of more than 400 students from fourth and fifth grades 

Effectiveness Lessons in Character was found to have potentially positive effects on academic achievement. It had no discernible effect on behav- 



ior or on knowledge, attitudes 


, and values. 








Behavior 


Knowledge, attitudes, and values 


Academic achievement 


Rating of effectiveness 


No discernible effects 


No discernible effects 


Potentially positive effects 


Improvement index 2 


Average: +8 percentile 
points 

Range: +8 percentile points 


Average: +6 percentile points, 
Range: +4 to +12 percentile points 


Average: +16 percentile points 
Range: +12 to +18 percentile 
points 



1. The evidence presented in this report is based on the available research. Findings and conclusions may change as new research becomes available. 

2. These numbers show the average and the range of improvement indices for all readings across the two studies. 







Additional program 
information 



Research 
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Developer and contact 

Lessons in Character was developed by Dr. David Brooks. Distri- 
bution is through Young People’s Press, 3033 5th Avenue, Suite 
200, San Diego, CA 92103. Web: www.youngpeoplespress.com. 
Email: info@youngpeoplespress.com. Telephone: 800-231-9774 
(M-F, 8-5 PST). 

Scope of use 

Information is not available on the scope of use. Lessons in Char- 
acter may have changed since the studies were conducted. The 
WWC recommends asking the developer for information about the 
most current version of this curriculum and taking into account that 
student demographics and school context may affect outcomes. 

Teaching 

The core curriculum for Lessons in Character consists of 24 
lessons at each grade level. A “theme of the week” approach 
is used with stories, activities, daily and weekly writing assign- 
ments, and a class project. Teachers introduce the week’s 
concept, read stories portraying that value in action, and engage 
students in targeted activities. Materials for students are avail- 
able in English or Spanish. 



Two studies reviewed by the WWC investigated the effects of 
Lessons in Character. One study (Dietsch, Bayha, & Zheng, 
2005) was a randomized controlled trial that met WWC evidence 
standards. The other study (DeVargas, 1998) was a quasi- 
experimental design that met WWC evidence standards with 
reservations. 

The Dietsch, Bahya, and Zheng study included 372 fourth- 
grade students from eight schools in Louisiana and Florida. 



The developer provides teaching resources such as bind- 
ers with strategies, skills, suggestions, and techniques for 
teaching Lessons in Character. The teacher’s guide provides 
the lesson’s main idea, concrete objectives, and teaching 
strategies, and suggests that class projects center on the les- 
sons’ main ideas. Supplemental curriculum materials are also 
available for teachers to use in daily oral language development 
and weekly writing assignments. The materials are designed to 
enhance students’ understanding of that week’s core concept. 
Teachers are encouraged to attend a training workshop held 
by the developer, study author, or school staff trained by the 
developer. Training workshops may vary from half a day to two 
full days. 

Cost 

The costs of materials for a classroom — including books, teach- 
er’s edition, black line masters, posters, and so forth — range 
from about $195 to $370 depending on the grade level. An 
administrator’s package is available for $139. According to the 
developer, the cost of a training workshop is $500 per day and 
may be individually negotiated based on the amount of materials 
purchased. 



It compared outcomes for students in 11 randomly assigned 
classrooms using the curriculum against the outcomes for 
students that did not use the curriculum in 10 control classrooms 
in the same schools. The DeVargas study included 61 fifth-grade 
students drawn from nine schools using the curriculum and 
six comparison schools in Fort Worth, Texas. The two studies 
focused on Lessons in Character as implemented in classrooms 
rather than as a schoolwide intervention. 
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Effectiveness 



The WWC found Lessons in 
Character to have potentially 
positive effects on academic 
achievement, and no 
discernible effects on 
behavior or on knowledge, 
attitudes, and values 
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Findings 

The WWC review of character education addresses student 
outcomes in three domains: behavior; knowledge, attitudes, and 
values; and academic achievement. 

Behavior. The Dietsch, Bayha, & Zheng study reported no 
differences in effects on citizenship grades. 

Knowledge, attitudes, and values. In the Dietsch, Bayha, & 
Zheng study students using Lessons in Character gave more 
positive responses to some of the items (such as do not pick 
on other students, show good sportsmanship) in four scales of 
character-related attitudes and values. The average response 
across each of these four scales was neither statistically 
significant (as calculated by the WWC) nor large enough to be 
substantively important using WWC criteria. 3 

The DeVargas study reported neither statistically significant 
nor substantively important differences between the Lessons in 
Character group and the comparison groups on a measure of 
moral development. 



Academic achievement. The Dietsch, Bayha, & Zheng study 
reported statistically significant differences favoring the Lessons 
in Character group on academic achievement (reading and math) 
and attendance (absences). Two of these outcomes (math and 
attendance) were found to be statistically significant (as calcu- 
lated by the WWC). 3 

Rating of effectiveness 

The WWC rates interventions as positive, potentially positive, 
mixed, no discernible effects, potentially negative, or negative. 
The rating of effectiveness takes into account four factors: the 
quality of the research design, the statistical significance of the 
findings (as calculated by the WWC), the size of the difference 
between participants in the intervention condition and the 
comparison conditions, and the consistency in findings across 
studies (see the WWC Intervention Rating Scheme) . 



Improvement index 

For each outcome domain, the WWC computed an improvement 
index based on the average effect size (see the WWC Improve- 
ment Index Technical Paper) . This improvement index represents 
the difference between the percentile rank of the average 
student in the intervention condition versus the percentile rank 
of the average student in the comparison condition. Unlike the 
rating of effectiveness, the improvement index is entirely based 
on the size of the effect, regardless of the statistical significance 
of the effect, the study design, or the analysis. The improvement 
index can take on values between -50 and +50, with positive 
numbers denoting favorable results. The average improvement 
index for behavior is +8 (Dietsch, Bayha, & Zheng, 2005). The 
average improvement index for knowledge, attitudes, and values 
in the two studies that WWC reviewed is +6 percentile points, 



with a range of +4 to +12 percentile points across findings. The 
average improvement index for academic achievement is +16 
percentile points, with a range of +12 to +18 percentile points 
(Dietsch, Bayha, & Zheng, 2005). 

Summary 

The WWC reviewed two studies on Lessons in Character. These 
studies found no discernible effects in behavior or in knowledge, 
attitudes, and values although the improvement index favored 
Lessons in Character in both cases. One study reported statisti- 
cally significant effects on academic achievement, which gener- 
ated a rating of potentially positive effects. Character education, 
an evolving field, is beginning to establish a research base. The 
evidence presented in this report is limited and may change as 
new research emerges. 



3. The level of statistical significance was calculated by the WWC and corrects for clustering within classrooms or schools, and for multiple comparisons. 
For an explanation see the WWC Tutorial on Mismatch . See the WWC Intervention Rating Scheme for the formulas the WWC used to calculate statistical 
significance. 
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Appendix 



Appendix A1.1 


Study characteristics: Dietsch, Bayha, & Zheng, 2005 (randomized controlled trial) 


Characteristic 


Description 


Study citation 


Dietsch, B., Bayha, J. L., & Zheng, H. (2005, April). Short-term effects of a character education program among fourth grade students. Paper presented at the American 
Educational Research Association annual meeting, Montreal, Quebec, Canada. 


Participants 


More than 400 fourth-grade students attending elementary schools in Louisiana and Florida. Eight schools whose principals expressed an interest in implementing the 
program were selected at random to participate. In each of the eight schools two fourth-grade teachers were randomly assigned to the intervention or control condition. Of the 
students in the study, 75% participated in the National School Lunch Program and about 50% met state reading and mathematics standards. The authors noted that all grade 
levels in the schools fell below state expectations in core academic areas. 


Setting 


All participating schools were located in rural, poor, and ethnically diverse communities in Louisiana and Florida. 


Intervention 


Teachers were encouraged to complete at least one program lesson per week for 1 5 weeks. The authors note that comprehensive character education would involve 
administrators, parents, and others on campus; this study used only the materials for classrooms. School officials were reluctant to have researchers enter the classroom, so 
fidelity and duration of student exposure to the intervention were not measured. 


Comparison 


Teachers in the comparison group were asked to teach as they normally would. They did not institute any new character education instruction during the study. After the 
posttests for the study, the teachers in the comparison group received copies of the classroom materials used in the program. 


Primary outcomes 
and measurement 


Outcome measures are based on student surveys and archival data (for attendance, citizenship, grades). The outcomes include mathematics, reading, and citizenship grades 
from report cards; attendance rates; positive and negative student behavior reflecting core values; and student attitudes toward core values. (See Appendices A2.1, A2.2, and 
A2.3.) 1 


Teacher training 


The authors characterized teacher training as "minimal.” Teachers who implemented the intervention received lesson materials and training. Each classroom assigned to the 
intervention group received two visits from a trainer, who observed the students and offered assistance to the teacher. 



1. Additional teacher-reported outcomes in the behavior domain (i.e., student behavior) could not be reviewed because the standard deviations reported by the study authors represented variations among teachers rather than variations 
among students. Therefore, a student-level effect size could not be estimated for this outcome. Further, the measure, teacher opinions of character education implementation, was not included in this review because it did not assess 
student outcomes. 
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Appendix A1.2 Study characteristics: DeVargas 1998 (quasi-experimental design) 



Characteristic 


Description 


Study citation 


DeVargas, R. C. (1998). A study of Lessons in Character: The effect of moral development curriculum upon moral judgment. Dissertation Abstracts International, 59 (11 -A), 
4042. (UMI No. 9913706) 


Participants 


Fifth-grade students selected from nine intervention and six comparison schools, with 21 classrooms and 31 students from the intervention schools and 17 classrooms and 
30 students from the comparison schools. No information was reported on the sample’s racial/ethnic composition or socioeconomic characteristics. 


Setting 


The study was conducted in Fort Worth Independent School District in Fort Worth, Texas. 


Intervention 


Teachers provided the instruction with the help of the school counselor, who acted as a trainer/facilitator. In some cases the counselor alone instructed the various classes 
in his/her school on a rotational basis. No information is reported about the fidelity of the implementation other than the author's assumption that the implementation of the 
Lessons In Character curriculum was similar across classrooms and across schools. 


Comparison 


The comparison schools were selected from the same geographic area as the intervention schools. Detailed information on the comparison condition was not provided. 


Primary outcomes 
and measurement 


The only outcome investigated in this study was moral judgment, as measured by the Sociomoral Reflection Measure — Short Form. (See Appendices A2.1, A2.2, and A2.3.) 


Teacher training 


School counselors acted as trainers or facilitators to the teachers during the implementation period. 
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Appendix A2.1 Outcome measures in the behavior domain 



Outcome measure 


Description 


Citizenship grades 


Proportion changing from below to above “satisfactory” on student report card (as cited in Dietsch, Bayha, & Zheng, 2005). 



Appendix A2.2 Outcome measures in the knowledge, attitudes, and values domain 



Outcome measure 


Description 


Student respect 


Student survey: The WWC aggregated the nine item-level results for a modified version of the subscales of the School as a Caring Community Profile (SCCP-II) by 
averaging their means and effect sizes (as cited in Dietsch, Bayha, & Zheng, 2005). 


Friendship and belonging 


Student survey: The WWC aggregated the eight item-level results for a modified version of the subscales of the School as a Caring Community Profile (SCCP-II) by 
averaging their means and effect sizes (as cited in Dietsch, Bayha, & Zheng, 2005). 


Shaping the environment 


Student survey: The WWC aggregated the seven item-level results for a modified version of the subscales of the School as a Caring Community Profile (SCCP-II) by 
averaging their means and effect sizes (as cited in Dietsch, Bayha, & Zheng, 2005). 


Support and care by 
teachers and staff 


Student survey: The WWC aggregated the four item-level results for a modified version of the subscales of the School as a Caring Community Profile (SCCP-II) by 
averaging their means and effect sizes (as cited in Dietsch, Bayha, & Zheng, 2005). 


Moral development 


Gibbs’ Sociomoral Reflection Measure-Short Form (SRM). 



Appendix A2.3 Outcome measures in the academic achievement domain 



Outcome measure 


Description 


Reading grades 


Grade point average in reading (as cited in Dietsch, Bayha, & Zheng, 2005). 


Mathematics grades 


Grade point average in mathematics (as cited in Dietsch, Bayha, & Zheng, 2005). 


Attendance (absences) 


Number of absences over the course of one semester (as cited in Dietsch, Bayha, & Zheng, 2005). 
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Appendix A3.1 Summary of study findings included in the rating for the behavior domain 3 









Author’s findings from the study 














Mean outcome 11 
(standard deviation 6 ) 




WWC calculations 




Outcome measure 11 


Study 

sample 


Sample size 
(classrooms/ 
students) 


Lessons in Comparison 

Character group group 

(column 1) (column 2) 


Mean difference 6 
(column 1- 
column 2) 


Statistical 
significance 9 
Effect size* (at a = 0.05) 


Improvement 

index h 



Dietsch, Bayha, & Zheng, 2005 (randomized controlled trial) 



Citizenship grades 


Grade 4 


4/141 


0.12 

(0.32) 


0.06 

(0.23) 


0.06 


0.21 


ns 


+8 


Domain average' for behavior 












0.21 


ns 


+8 



a. This appendix reports findings considered for the effectiveness rating and the improvement index. 

b. This appendix reports unadjusted posttest means reported by the study authors and the covariate-adjusted mean differences estimated by the WWC based on information from the study authors for the purpose of effect size calculation. 

c. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. The WWC 
requested and received standard deviations for all outcomes in the first study, Dietsch, Bayha, & Zheng (2005), because they were not reported in the paper. 

d. Additional teacher-reported outcomes in the behavior domain (i.e., student behavior) could not be reviewed because the standard deviations reported by the study authors represented variations among teachers rather than variations 
among students. Therefore, a student-level effect size could not be estimated for this outcome. Further, the measure, teacher opinions of character education implementation, was not included in this review because it did not assess 
student outcomes. For a description of the types of outcome measures included in this topic review see the Character Education Abstract . 

e. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. The measure, absences, was reversed so that a positive difference would favor the intervention group 
in Dietsch, Bayha, & Zheng (2005). 

f. For an explanation of the effect size calculation, please see the WWC Technical Working Paper on Effect Size . 

g. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between groups. The level of statistical significance was calculated by the WWC and where necessary, cor- 
rects for clustering within classrooms or schools, and for multiple comparisons. For an explanation, see the WWC lutorial on Mismatch . See the WWC Intervention Rating Scheme for the formulas the WWC used to calculate the statisti- 
cal significance. In the case of the Lessons in Character report, a correction for clustering was needed. 

h. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values 
between -50 and +50, with positive numbers denoting favorable results. 

i. This row provides the study average, which is also the domain average in this case. The WWC-computed domain average effect size is a simple average rounded to two decimal places. The domain improvement index is calculated from 
the average effect size. 
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Appendix A3.2 Summary of study findings included in the rating for the knowledge, attitudes, and values domain 3 









Author’s findings from the study 














Mean outcome 11 
(standard deviation 6 ) 




WWC calculations 




Outcome measure 11 


Study 

sample 


Sample size 
(classrooms/ 
students) 


Lessons in Comparison 

Character group group 

(column 1) (column 2) 


Mean difference 6 
(column 1- 
column 2) 


Statistical 
significance 9 
Effect size 1 (at a = 0.05) 


Improvement 

index 11 



Dietsch, Bayha, & Zheng, 2005 (randomized controlled trial) 


Student respect 


Grade 4 


21/372 


3.30 

(na) 


3.11 

(na) 


0.19 


0.13 


ns 


+5 


Friendship and belonging 


Grade 4 


21/372 


3.22 

(na) 


2.97 

(na) 


0.25 


0.18 


ns 


+7 


Shaping the environment 


Grade 4 


21/372 


3.04 

(na) 


2.70 

(na) 


0.34 


0.24 


ns 


+9 


Support and care by 
teachers and staff 


Grade 4 


21/372 


4.11 

(na) 


3.70 

(na) 


0.41 


0.30 


ns 


+12 


Average' for knowledge, attitudes, values (Dietsch, Bayha, & Zheng, 2005) 






0.21 


ns 


+8 


DeVargas, 1998 (quasi-experimental study) 


Moral development 


Grade 5 


15 schools/ 


2.20 


2.17 


0.03 


0.11 


ns 


+4 






61 students 


(0.29) 


(0.27) 










Average' for knowledge, attitudes, values (DeVargas, 1998) 








0.11 


ns 


+4 


Domain average' for knowledge, attitudes, and values across all studies 








0.16 




+6 


na = not applicable 


















ns = not statistically significant 


















a. This appendix reports findings considered for the effectiveness rating and the improvement index. The WWC averaged individual items to provide the scales reported here for four outcomes (student respect, friendship and belonging, 



shaping the environment, and support and care by teachers and staff). Findings on the individual item level are presented in Appendix A4. 

b. This appendix reports posttest unadjusted means reported by the study authors and the covariate-adjusted mean differences estimated by the WWC based on information from the study authors for the purpose of effect size calculation. 

c. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. The WWC 
requested and received standard deviations for all outcomes in the first study, Dietsch, Bayha, & Zheng (2005), because they were not reported in the paper. These standard deviations were used to calculate effect size on the individual 
item level. (See Appendix A4.) Therefore, standard deviations on the scale level are presented as “not applicable” in this appendix. 

d. Additional teacher-reported outcomes in the knowledge, attitudes, and values (i.e., student knowledge and reasoning; student values) domain could not be reviewed. The standard deviations reported by the study authors represented 
variations among teachers rather than variations among students. Therefore, a student effect size could not be estimated for this outcome. Further, the measure, teacher opinions of character education implementation, was not included 
in this review because it did not assess student outcomes. For a description of the types of outcome measures included in this topic review see the Character Education Abstract . 

e. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. The measure, absences, was reversed so that a positive difference would favor the intervention group 
in Dietsch, Bayha, & Zheng (2005). 

f. For an explanation on effect size calculation, please see the WWC Technical Working Paper on Effect Size . 

g. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between groups. The level of statistical significance was calculated by the WWC and where necessary, cor- 
rects for clustering within classrooms or schools, and for multiple comparisons. For an explanation, see the WWC Tutorial on Mismatch . See the WWC Intervention Rating Scheme for the formulas the WWC used to calculate the statisti- 
cal significance. In the case of the Lessons in Character repori, corrections for clustering and multiple comparisons were needed. 

h. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values 
between -50 and +50, with positive numbers denoting favorable results. 

i. The WWC-computed average effect sizes for each study and for the domain across studies are simple averages rounded to two decimal places. The average improvement indices are calculated from the average effect sizes. 
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Appendix A3.3 Summary of study findings included in the rating for the academic achievement domain 3 









Author’s findings from the study 














Mean outcome b 
(standard deviation 0 ) 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(classrooms/ 
students) 


Lessons in Comparison 

Character group group 

(column 1) (column 2) 


Mean difference 11 
(column 1- 
column 2) 


Statistical 
significance' 
Effect size" (at a = 0.05) 


Improvement 

index 9 



Dietsch, Bayha, & Zheng, 2005 (randomized controlled trial) 



Reading grades 


Grade 4 


4/141 


2.95 

(0.88) 


2.66 

(0.99) 


0.29 


0.31 


ns 


+12 


Mathematics grades 


Grade 4 


4/141 


2.95 

(0.84) 


2.56 

(0.86) 


0.39 


0.46 


Statistically 

significant 


+18 


Attendance (absences) 


Grade 4 


4/141 


1.46 

(1.93) 


2.59 

(2.72) 


-1.13 


0.48 


Statistically 

significant 


+18 


Domain average 11 for academic achievement 










0.41 


Statistically 

significant 


+16 



ns = not statistically significant 



a. This appendix reports findings considered for the effectiveness rating and the improvement index. 

b. This appendix reports unadjusted posttest means reported by the study authors and the covariate-adjusted mean differences estimated by the WWC based on information from the study authors for the purpose of effect size calculation. 

c. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. The WWC 
requested and received standard deviations for all outcomes in the first study, Dietsch, Bayha, & Zheng (2005), because they were not reported in the paper. 

d. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. The measure, absences, was reversed so that a positive difference would favor the intervention group 
in Dietsch, Bayha, & Zheng (2005). 

e. For an explanation on effect size calculation, please see the WWC Technical Working Paper on Effect Size . 

f. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between groups. The level of statistical significance was calculated by the WWC and where necessary, cor- 
rects for clustering within classrooms or schools, and for multiple comparisons. For an explanation, see the WWC Tutorial on Mismatch . See the WWC Intervention Rating Scheme for the formulas the WWC used to calculate the statisti- 
cal significance. In the case of the Lessons in Character report, corrections for clustering and multiple comparisons were needed. 

g. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values 
between -50 and +50, with positive numbers denoting favorable results. 

h. This row provides the study average, which is also the domain average in this case. The WWC-computed domain average effect size is a simple average rounded to two decimal places. The domain improvement index is calculated from 
the average effect size. 
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Appendix A4 Summary of detailed study findings in the knowledge, attitudes, and values domain 3 









Author’s findings from the study 














Mean outcome b 
(standard deviation^ 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(classrooms/ 
students) 


Lessons in Comparison 

Character group group 

(column 1) (column 2) 


Mean difference 11 
(column 1- 
column 2) 


Statistical 
significance' 
Effect size" (at a = 0.05) 


Improvement 

index 9 



Dietsch, Bayha, & Zheng, 2005 (randomized controlled trial) 



Student respect 



Treat classmates with respect 


Grade 4 


21/372 


2.97 

(1.36) 


2.90 

(1.29) 


0.07 


0.05 


ns 


+2 


Respect personal property of 
others 


Grade 4 


21/372 


3.09 

(1.49) 


3.28 

(1.48) 


-0.19 


-0.13 


ns 


-5 


Show respect for school 
property 


Grade 4 


21/372 


3.13 

(1.55) 


2.98 

(1.44) 


0.15 


0.10 


ns 


+4 


Behave respectfully toward 
school staff 


Grade 4 


21/372 


3.41 

(1.45) 


3.31 

(1.33) 


0.10 


0.08 


ns 


+3 


Show respect toward teachers 


Grade 4 


21/372 


3.68 

(1.54) 


3.43 

(1.38) 


0.25 


0.18 


ns 


+7 


Do not pick on other students 


Grade 4 


21/372 


3.20 

(1.58) 


2.64 

(1.48) 


0.56 


0.37 


ns 


+14 


Show good sportsmanship 


Grade 4 


21/372 


3.87 

(1.40) 


3.39 

(1.44) 


0.48* 


0.33 


ns 


+13 


Show respect toward other 
students 


Grade 4 


21/372 


3.73 

(1.35) 


3.27 

(1.32) 


0.46* 


0.35 


ns 


+14 


Do not put others down 


Grade 4 


21/372 


2.63 

(1.45) 


2.77 

(1.27) 


-0.14 


-0.11 


ns 


-4 



Friendship and belonging 



Comfort others who are sad 


Grade 4 


21/372 


3.10 

(1.47) 


2.90 

(1.33) 


0.20 


0.14 


ns 


+6 


Help each other even if not 
friends 


Grade 4 


21/372 


2.95 

(1.43) 


2.68 

(1.32) 


0.27 


0.19 


ns 


+8 


Work well with each other 


Grade 4 


21/372 


3.35 

(1.33) 


3.36 

(1.31) 


-0.01 


0.00 


ns 


+0 



(continued) 
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Appendix A4 Summary of detailed study findings in the knowledge, attitudes, and values domain 3 (continued) 









Author’s findings from the study 














Mean outcome b 
(standard deviation 0 ) 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(classrooms/ 
students) 


Lessons in Comparison 

Character group group 

(column 1) (column 2) 


Mean difference 11 
(column 1- 
column 2) 


Statistical 
significance' 
Effect size" (at a = 0.05) 


Improvement 

index 9 



Help new students feel accepted 


Grade 4 


21/372 


3.61 


3.39 


0.22 


0.16 


ns 


+6 








(1.42) 


(1.37) 










Be willing to forgive each other 


Grade 4 


21/372 


3.33 


2.87 


0.46 


0.31 


ns 


+12 








(1.52) 


(1.36) 










Be patient with each other 


Grade 4 


21/372 


3.04 


2.66 


0.38 


0.29 


ns 


+12 








(1.34) 


(1.26) 










Listen to each other in the 


Grade 4 


21/372 


3.13 


2.93 


0.20 


0.14 


ns 


+5 


classroom 






(1.49) 


(1.39) 










Share what they have with 


Grade 4 


21/372 


3.25 


2.93 


0.32 


0.22 


ns 


+9 


others 






(1.47) 


(1.40) 










Shaping the environment 


Try to make it up if do something 


Grade 4 


21/372 


3.15 


2.89 


0.26 


0.18 


ns 


+7 


harmful 






(1.45) 


(1.39) 










Get other students to follow 


Grade 4 


21/372 


2.95 


2.69 


0.26 


0.18 


ns 


+7 


school rules 






(1.53) 


(1.29) 










Help to improve the school 


Grade 4 


21/372 


3.14 


2.99 


0.15 


0.11 


ns 


+4 


(1.56) 


(1.30) 




Have a positive influence on 


Grade 4 


21/372 


3.04 


2.67 


0.37 


0.27 


ns 


+11 


behavior 






(1.43) 


(1.31) 










Resolve conflicts without fighting 


Grade 4 


21/372 


2.87 


2.48 


0.39 


0.27 


ns 


+10 


or insults 






(1.50) 


(1.43) 










Stop it when see others being 


Grade 4 


21/372 


2.95 


2.57 


0.38 


0.26 


ns 


+10 


picked on 






(1.53) 


(1.31) 










Help to solve school problems 


Grade 4 


21/372 


3.15 


2.58 


0.57 


0.38 


ns 


+15 








(1.56) 


(1.39) 










Support and care by teachers and staff 


Can talk to teachers about their 


Grade 4 


21/372 


3.84 


3.52 


0.32 


0.23 


ns 


+9 


problems 






(1.37) 


(1.40) 











(continued) 
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Appendix A4 Summary of detailed study findings in the knowledge, attitudes, and values domain 3 (continued) 









Author’s findings from the study 














Mean outcome b 
(standard deviation 0 ) 




WWC calculations 








Sample size 


Lessons in 


Comparison 


Mean difference 11 


Statistical 






Study 


(classrooms/ 


Character group 


group 


(column 1- 


significance' 


Improvement 


Outcome measure 


sample 


students) 


(column 1) 


(column 2) 


column 2) 


Effect size" (at a = 0.05) 


index 9 



Teachers help students who 
need extra help 


Grade 4 


21/372 


4.47 

(1.26) 


3.65 

(1.45) 


0.82 


0.61 


Statistically 

significant 


+23 


Adults can be counted on to 


Grade 4 


21/372 


4.11 


3.93 


0.18 


0.14 


ns 


+5 


make sure that students are safe 






(1.31) 


(1.34) 










Teachers are fair in treatment of 


Grade 4 


21/372 


4.03 


3.70 


0.33 


0.24 


ns 


+9 


students 






(1.38) 


(1.43) 











a. This appendix presents item-level findings for four scales in the knowledge, attitudes, and values domain: student respect, friendship and belonging, shaping the environment, and support and care by teachers and staff. Aggregated 
scale scores were used for rating purposes and are presented in Appendix A3. 2. 

b. This appendix reports posttest unadjusted means reported by the study authors. 

c. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. 

d. Covariate-adjusted mean differences estimated by the WWC based on information from the study authors for the purpose of effect size calculation. Positive differences favor the intervention group: negative differences favor the com- 
parison group. 

e. For an explanation on effect size calculation, please see the WWC Technical Working Paper on Effect Size . 

f. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between groups. The level of statistical significance was calculated by the WWC and where necessary, cor- 
rects for clustering within classrooms or schools. For an explanation, see the WWC Tutorial on Mismatch . See the WWC Intervention Rating Scheme for the formulas the WWC used to calculate the statistical significance. In the case of 
the Lessons in Character report, a correction for clustering was needed. 

g. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values 
between -50 and +50, with positive numbers denoting favorable results. 
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Appendix A5.1 Rating for the behavior domain 



The WWC rates interventions as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative. 1 

For the outcome domain of behavior, the WWC rated Lessons in Character as having no discernible effects. It did not meet the criteria for positive effects, because 
it only had one study. In addition, it did not meet the criteria for other ratings (potentially positive effects, mixed effects, potentially negative effects, negative effects) 
because none of the studies showed statistically significant or substantively important effects. 

Rating received 

No discernible effects: No affirmative evidence of effects. 

• Criterion 1 : None of the studies shows a statistically significant or substantively important effect, either positive or negative. 

Met. The WWC analysis found no statistically significant or substantively important effects in this domain. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. Lessons in Character had only one evaluation study meeting WWC evidence standards that reported findings on behavior, and so did not 
meet this criterion. Further, although that one study used a strong design, it did not show statistically significant positive effects. 

• Criterion 2: No studies showing statistically significant or substantively important negative effects. 

Met. The WWC analysis found no statistically significant or substantively important negative effects in this domain. 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect, thus qualifying as a positive effect. 

Not met. The WWC analysis found no statistically significant or substantively important positive effects in this domain. 

• Criterion 2: No studies showing a statistically significant or substantively important negative effect. Fewer or the same number of studies showing indeterminate 
effects than showing statistically significant or substantively important positive effects. 

Not met. No studies showed a statistically significant or substantively important negative effect. Because one study showed indeterminate effects 
and no studies showed statistically significant or substantively important positive effects, Lessons in Character did not meet this criterion. 

(continued) 



WWC Intervention Report 



10 









Appendix A5.1 Rating for the behavior domain (continued) 



Mixed effects: Evidence of both positive and negative effects. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect. At least one study showing a statistically significant or 
substantively important negative effect, but no more such studies than the number showing a statistically significant or substantively important positive effect. 

Not met. The WWC analysis found no statistically significant or substantively important positive or negative effects in this domain. 

• Criterion 2: At least one study showing a statistically significant or substantively important effect, and more studies showing an indeterminate effect than showing 
a statistically significant or substantively important effect. 

Not met. The WWC analysis found no statistically significant or substantively important effects in this domain. 

Potentially negative effects: Evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1: At least one study showing a statistically significant or substantively important negative effect. No studies showing a statistically significant or 
substantively important positive effect, or more studies showing statistically significant or substantively important negative effects than showing statistically 
significant or substantively important positive effects. 

Not met. The WWC analysis found no statistically significant or substantively important negative effects in this domain. 

Negative effects: Strong evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant negative effects, at least one of which is based on a strong design. 

Not met. The WWC analysis found no statistically significant negative effects in this domain. 

• Criterion 2: No studies showing statistically significant or substantively important positive effects. 

Met. The WWC analysis found no statistically significant or substantively important positive effects in this domain. 



1. For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain level effect. The WWC also considers the size of the domain level effect for ratings of 
potentially positive effects. See the WWC Intervention Rating Scheme for a complete description. 
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Appendix A5.2 Rating for the knowledge, attitudes, and values domain 



The WWC rates interventions as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative. 1 

For the outcome domain of knowledge, attitudes, and values, the WWC rated Lessons in Character as having no discernible effects. It did not meet the criteria for 
other ratings (positive effects, potentially positive effects, mixed effects, potentially negative effects, and negative effects) because none of the studies showed statisti- 
cally significant or substantively important effects. 



Rating received 

No discernible effects: No affirmative evidence of effects. 

• Criterion 1: None of the studies shows a statistically significant or substantively important or statistically significant effect, either positive or negative. 

Met. The WWC analysis found no statistically significant effects in this domain. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. Lessons in Character had two evaluation studies meeting WWC evidence standards that reported findings on knowledge, attitudes, and 
values. One of those studies used a strong design. But none of the studies showed a statistically significant or substantively important positive 
effect, so Lessons in Character did not meet this criterion. 

• Criterion 2: No studies showing statistically significant or substantively important negative effects. 

Met. The WWC analysis found no statistically significant or substantively important negative effects in this domain. 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect, thus qualifying as a positive effect. 

Not met. The WWC analysis found no statistically significant or substantively important positive effects in this domain. 

• Criterion 2: No studies showing a statistically significant or substantively important negative effect. Fewer or the same number of studies showing indeterminate 
effects than showing statistically significant or substantively important positive effects. 

Not met. No studies showed a statistically significant or substantively important negative effect. Because one study showed indeterminate effects 
and no studies showed statistically significant or substantively important positive effects, Lessons in Character did not meet this criterion. 

(continued) 
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Appendix A5.2 Rating for the knowledge, attitudes, and values domain (continued) 



Mixed effects: Evidence of both positive and negative effects. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect. At least one study showing a statistically significant or 
substantively important negative effect, but no more such studies than the number showing a statistically significant or substantively important positive effect. 

Not met. The WWC analysis found no statistically significant or substantively important positive or negative effects in this domain. 

• Criterion 2: At least one study showing a statistically significant or substantively important effect, and more studies showing an indeterminate effect than showing 
a statistically significant or substantively important effect. 

Not met. The WWC analysis found no statistically significant or substantively important effects in this domain. 

Potentially negative effects: Evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1: At least one study showing a statistically significant or substantively important negative effect. No studies showing a statistically significant or 
substantively important positive effect. The number of studies showing statistically significant or substantively important negative effects is greater than the 
number showing statistically significant or substantively important positive effects. 

Not met. The WWC analysis found no statistically significant or substantively important negative effects in this domain. 

Negative effects: Strong evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant negative effects, at least one of which is based on a strong design. 

Not met. The WWC analysis found no statistically significant negative effects in this domain. 

• Criterion 2: No studies showing statistically significant or substantively important positive effects. 

Met. The WWC analysis found no statistically significant or substantively important positive effects in this domain. 



1. For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain level effect. The WWC also considers the size of the domain level effect for ratings of 
potentially positive effects. See the WWC Intervention Rating Scheme for a complete description. 
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Appendix A5.3 Rating for the academic achievement domain 



The WWC rates interventions as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative. 1 

For the outcome domain of academic achievement, the WWC rated Lessons in Character as having no discernible effects. It did not meet the criteria for positive 
effects because none of the studies showed statistically significant or substantively important effects. The remaining ratings (mixed effects, no discernible effects, 
potentially negative effects, and negative effects) were not considered because Lessons in Character was assigned the highest applicable rating. 

Rating received 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect, thus qualifying as a positive effect. 

Met. In the one study on Lessons in Character that examined student academic outcomes, the average effect size was positive and statistically 
significant. Further, effects on two student outcomes were positive and statistically significant. 2 

• Criterion 2: No studies showing a statistically significant or substantively important negative effect. The number of studies showing indeterminate effects is not 
greater than the number showing statistically significant or substantively important positive effects. 

Met. No studies showed a statistically significant or substantively important negative effect. Because one study showed statistically significant 
positive effects and no studies showed indeterminate effects, Lessons in Character met this criterion. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. Lessons in Character had only one evaluation study meeting WWC evidence standards that reported findings on academic achievement. 

But that one study used a strong design and statistically significant positive effects. 

• Criterion 2: No studies showing statistically significant or substantively important negative effects. 

Met. The WWC analysis found no statistically significant or substantively important negative effects in this domain. 



1. For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain level effect. The WWC also considers the size of the domain level effect for ratings of 
potentially positive effects. See the WWC Intervention Rating Scheme for a complete description. 

2. Although the study author reported three statistically significant effects, the WWC analysis confirmed the significance of only two of those findings. 



WWC Intervention Report 



14 








